A Topological View of Unsupervised Learning from Noisy Data

In this paper, we take a topological view of unsupervised learning. From this point of view, clustering may be interpreted as trying to find the number of connected components of an underlying geometrically structured probability distribution in a certain sense that we will make precise. We construct a geometrically structured probability distribution that seems appropriate for modeling data in very high dimensions. A special case of our construction is the mixture of Gaussians where there is Gaussian noise concentrated around a finite set of points (the means). More generally we consider Gaussian noise concentrated around a low dimensional manifold and discuss how to recover the homology of this underlying geometric core from data that does not lie on it. We show that if the variance of the Gaussian noise is small in a certain sense, then the homology can be learned with high confidence by an algorithm that has a weak (linear) dependence on the ambient dimension. Our algorithm has a natural interpretation as a spectral learning algorithm using a combinatorial laplacian of a suitable data-derived simplicial complex. Weinberger was supported by DARPA and Smale, Niyogi were supported by the NSF while this work was conducted. Departments of Computer Science, Statistics, University of Chicago. Toyota Technological Institute, Chicago. Department of Mathematics, University of Chicago.